The present invention relates to a method for the transformation of a moving image sequence and further to a moving image sequence transformation device.
Camera devices with a digital image sensor for recording a video or a moving image sequence, i.e. a plurality of individual images successively recorded in a storage or a storage medium, are widely used today.
In such camera devices, such as a digital camera or a smartphone, different image sensor types are used with respect to the image detection technique.
In a CCD sensor, a two-dimensional individual image of a moving image sequence, i.e. extending along a horizontal and a vertical axis, which is oriented orthogonally to this horizontal axis, is detected at a predetermined point of time or moment, so that the image points for a respective individual image, which form the individual image, are assigned exactly to one single detection time.
In contrast to the CCD image sensor, when recording a moving image sequence with a CMOS sensor, each individual image of the moving image sequence is detected in image sections, in particular row by row or line by line, wherein two image rows following one after the other in the vertical direction can only be detected with a time offset, i.e. not at the exact same time due to the discharge inertia or slowness of the CMOS sensor. When using a CMOS sensor for the detection of moving image sequences, the rows of an individual image which follow one after the after are detected at different points of time. In contrast, within a respective horizontally extending row of a CMOS sensor, the information forming a row of the individual image are read at the same point of time, i.e. without time offset in the horizontal direction.
Detecting image information row by row at different points of time depending on the vertical row position within the respective individual image is generally problematic for recording a sequence of moved images, i.e. a moving image sequence. For example, if an object, which is recorded on multiple successive individual images in the moving image sequence, moves in the recorded individual images vertically from top to bottom, the object will initially be detected in vertically upward or higher image rows and thus relatively earlier points of time within a detection time interval assigned to the respective individual image. During the vertical downward movement of the object, the object will be detected at an increasingly relatively later point of time within the detection time interval of a respective individual image, because the object is detected in the respective individual image in an image row located further at the bottom.
When recording a moving image sequence with a CMOS sensor with row-based individual image detection, the recorded moving image sequence is often distorted. If, for example, a stationary object is detected from a moved car, for example a strictly vertically extending and stationary lamp post, the lamp post will be put obliquely in the respective individual images, and is thus recorded in a distorted manner that deviates from the strictly vertical orientation. In contrast, an exterior mirror of the car detected in the moving image sequence simultaneously for the above described case, is detected without such distortion in the respective individual images, when no relative movement takes place between the CMOS sensor recording the moving image sequence row by row and the external mirror. This distortion problem caused by recording individual images row by row is also referred to as rolling shutter problem.
A particular problem for image quality of a moving image sequence recorded with a CMOS sensor row by row are shaking or shocks of the CMOS sensor in relation to the recorded environment in the vertical direction. The timely offset row-by-row detection of the respective individual images, which extends across all images, does not only result in the distortion described above, but also in strong wobbling and compression of the individual images of the moving image sequence.